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Abstract. A group is said to have the Roo property if every automorphism has an infinite 
number of twisted conjugacy classes. We study the question whether G has the Rao property 
when G is a finitely generated torsion-free nilpotent group. As a consequence, we show that 
for every positive integer n > 5, there is a compact nilmanifold of dimension n on which every 
homeomorphism is isotopic to a fixed point free homeomorphism. As a by-product, we give 
a purely group theoretic proof that the free group on two generators has the R^o property. 
The Roo property for virtually abelian and for C-nilpotent groups are also discussed. 



1. Introduction 

A natural question with a long history in group theory is whether an infinite group must have 
an infinite number of conjugacy classes of elements. In 1949, Higman-Neumann-Neumann 
constructed an infinitely generated group with only a finite number of conjugacy classes. 
Subsequently, S. Ivanov constructed one such example where the group is finitely generated. 
More recently, D. Osin gave an example of a finitely generated infinite group in which all non- 
trivial elements are conjugate. More generally, given a group endomorphism (y9 : tt ^ vr, one 
considers the yj-twisted conjugacy classes. Equivalently, vr acts on tt via a-a ^ (ya(^{(y)~^ and 
the Reidemeister number R{ip), the cardinality of the set of orbits of this action, is the same as 
the number of (/^-twisted conjugacy classes. Clearly, -R(id^) is simply the number of conjugacy 
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classes. From the point of view of Nielsen- Wecken fixed point theory, R{ip) is precisely the 
number of fixed point classes of any map f : X X with induced homomorphism f# = (p 
on 7ri(X) = IT. For many spaces X (e.g. X is a compact manifold of dimension at least 
three), the Nielsen number N{f), which is the principal object of study, coincides with the 
minimal number MF[f] of fixed points of maps within the homotopy class of /. The Nielsen 
number is always bounded above by the Reidemeister number R{f) = R{<f) and for a large 
class of spaces, N{f) = or A^(/) = R{f)- While the computation of A^(/) is in general 
very difficult, the problem of determining the finiteness of R{(p) is more tractable. In 1985, 
D. Anosov [T] and independently E. Fadell and S. Husseini [5] showed that for any selfmap 
f : N ^ N of a compact nilmanifold, = N{f) where L{f) denotes the Lefschetz 

number of /. Using the approach of [3], this was later strengthened in [10] where it was 
shown in particular that A^(/) > iff R{f) < oo. In fact, for selfmaps / of a nilmanifold, 
either N{f) = or A^(/) = R{f). 

In 1994, A. Fel'shtyn and R. Hill [9] conjectured that for a finitely generated group vr of 
exponential growth, if : tt ^ vr is injective then R{ip) = oo. Using techniques from 
geometric group theory, G. Levitt and M. Lustig [24J showed that if vr is finitely generated 
torsion- free non-elementary Gromov hyperbolic then every automorphism ip G Aut(7r) must 
have R{<f) = oo. This was subsequently proven in fB] without the torsion- free assumption. 
We say that a group G has the Roo property for automorphisms, in short G has the Roo 
property, if for every automorphism ip : G ^ G we have R{ip) = oo. It was shown in fT3] that 
the Fel'shtyn-Hill conjecture does not hold in general. Moreover, non-elementary polycyclic 
groups of exponential growth that are not Gromov hyperbolic without the R^o property were 
constructed. Since then, many examples of groups with R^o have been discovered (see e.g., 
[Zl, [H], [H], [IB], [22], [2S], and [23 )■ (For connections between the study of Reidemeister 
classes and other areas, see e.g. [12] •) In particular, groups that are quasi-isometric to 
generalized Baumslag-Solitar groups [29] have the -Roo property. In these examples, the 
groups are all of exponential growth. Since a result of M. Gromov states that a finitely 
generated group is of polynomial growth iff it is virtually nilpotent, it is natural to ask 
if one can find (virtually) nilpotent groups with the -Roo property. The main objective of 
this paper is to construct finitely generated groups of polynomial growth that have the -Roo 
property. This shows that the -Roo property does not depend on the growth type of the group 
as suggested by the original conjecture of Fel'shtyn and Hill. 
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It is easy to see that finitely generated abelian groups do not have the Roo property. There- 
fore, we will first explore the Roo property for virtually abelian and for nilpotent groups. 

The basic algebraic techniques used in the present paper for showing R{(p) = oo is the 
relationship among the Reidemeister numbers of groups homomorphisms of a short exact 
sequence. In general, given a commutative diagram of groups and homomorphisms 

A B 
A B 

the homomorphism rj induces a function fj : TZ{ip) T^if) where TZ{a) denotes the set of 
a-twisted conjugacy classes. Some of the basic facts that will be used throughout this paper 
are given in the following lemma. For more general results, see [12] and 



Lemma 1.1. Given an endomorphism ip : G G of a finitely generated torsion-free abelian 
group G, Fixip = {1} iff R{iIj) < oo. Consider the following commutative diagram 

1 > A > B > G > 1 



1 > A > B > G > 1 

where the rows are short exact sequences of groups and the vertical arrows are group endo- 
morphisms. 

(1) If R{}p) = oo then R{ip) = oo. 

(2) If R(Tp) < oo, \Fix'i^\ < oo, and R{(p') = oo then R{(p) = oo. 

(3) If the short exact sequence is a central extension then R{f) = R{ip')R{'^) . 

Proof. Suppose G is finitely generated torsion-free abelian. Then G = 1}^ for some positive 
integer k. For any ip : G ^ G, R{f) = i^Goker{l — ip) = \ det(l — f)\ so that R{(p) < oo iff 
(p does not have 1 as an eigenvalue iff ^plx) = x has only trivial solution, i.e., Fixip = 1. 

The homomorphism p : B ^ G induces a function p : IZ{ip) —>■ Hijp) given by p([q;]b) = 
[p(a;)]c- Since p is surjective, so is p. Thus, (1) follows. Similarly, i : A B induces a 
function i : 7l{ip') I^i'p)- Since the sequence is exact, it follows that i{Tl{(f')) = p~^{[l]c). 
The subgroup FixTp acts on IZ{lp') via 6 ■ [a]A = [dceLp{6)^^]A where 9 E B and 6 E FixTp. 
Thus, two classes [a\A and [(3\a are in the same F^x^-orbit iff and i{(3) are in the same 
Reidemeister class, i.e., = [i{l3)]B- Now, (2) follows immediately. Finally, if the 



extension is central, ^"^([ajc) is independent of d so that R{ip) = k ■ R{ip') and k is the 
number of distinct Reidemeister classes of Tp in C. In other words, k = R(jp) and thus (3) 
follows. □ 

In [3], J. Dyer constructed some nilpotent Lie algebras which have the property that every 
automorphism is unipotent. This implies that the induced homomorphism on the abelian- 
ization is the identity. This should imply that every automorphism of the corresponding 
nilpotent group has the property that the induced automorphism on the abelianization is 
the identity. An immediate consequence of this is the fact that every automorphism of 
such nilpotent group has an infinite number of yj-twisted conjugacy classes and thus the 
group has the Roo property. The following example is the group theoretic analog of the Lie 
algebra theoretic example constructed by Dyer [1] . 

Example 1.2. Let G be the quotient of the free nilpotent group F2/Tj{F2) on two generators 
x,y by the normal closure of the words Yf-^Y^, [Y2,x], [Yi,y]U4^^, [Yi, x][Y2,y]~^U^'^ . Here 
B = [x,y], Y, = [[B,y],y], Y2 = [[B,y],x], F3 = [[B,x],x], z, = [Y^y], ^2 = [Yux], 
= [Y2,y], f/4 = [-23, x]. Then given any automorphism if of G the induced homomorphism 
on the abelianization of G is the identity. Therefore, this group has the -Roo property. It has 
nilpotency class 6 and Hirsch length > 7. 

Remark 1.1. One can construct as in [2] a finitely generated torsion- free nilpotent group G 
so that Aut(G'), modulo the kernel of the action of Aut(G) on the abelianization G"^^, is a 
prescribed closed subgroup of GL(2,Z). 

This example has prompted us to determine which finitely generated torsion-free nilpotent 
groups can have the R^o property. In general, the R^o property has important implications 
in the fixed point theory of Jiang-type spaces since R{f) = 00 implies that A^(/) = 
which in turn implies in most cases that / is deformable to be fixed point free. Recall 
that a space is said to be of Jiang-type if L{f) = ^ ^(/) = and L{f) 7^ =^ 
A^(/) = R{f). Since nilmanifolds are known to be of Jiang-type, for each torsion- free 
nilpotent group which has the R^o property there corresponds a nilmanifold with the property 
that every homeomorphism is homotopic to a fixed point free map. Such examples cannot 
appear in dimension 3 but for every n > 3 there exists an n-dimensional nilmanifold such 
that every homeomorphism is homotopic to be fixed point free; and if n > 5 then the 

homotopy can be made to be isotopy. As a by-product of our investigation, we are able to 
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give an algebraic proof that automorphisms of free groups on two generators have infinite 
Reidemeister number. 

This paper is organized into seven sections. In section 2, extension of the Roc property to 
automorphisms of virtually abelian groups is discussed. In particular, we show that the 
fundamental group of the Klein bottle (Theorem 12.21) . which is a finitely generated torsion- 
free virtually abelian group, has the Roo property. We also construct for every n > 2 a 
finitely generated torsion-free virtually abelian group of Hirsch length n with the desired Roo 
property (Theorem 12. 4p . In section 3 we show that F2/Tk has the Roo property for k >9 
(Theorem 13.41) where F2 is the free group on two generators and = rk{F2) is the k-th term 
of the lower central series of F2. In sections 4 and 5, examples of finitely generated torsion- 
free nilpotent groups with the Roo property are presented, according to the nilpotency class 
and the Hirsch length. In particular, we show that for each n > 3, there is a nilpotent group 
of Hirsch length n with the Roo property. Furthermore, nilpotent groups of nilpotency class 
2 that have the -Roo property are also constructed. In the section 6, we turn to topological 
applications. We show that there exist nilmanifolds of dimension n for each > 5 on which 
every homeomorphism is isotopic to a fixed point free homeomorphism (Theorem 16. ip . In 
the last section, we consider a generalization of nilpotent groups, namely C-nilpotent groups 
where C denotes the class of finite groups. These groups provide further examples of groups 
with the Roo property. For any n > 7, we make use of the results from section 5 to construct 
a compact C-nilpotent manifold of dimension n for which every homeomorphism is isotopic 
to a fixed point free homeomorphism (Theorem 17. 3p . 

2. VIRTUALLY ABELIAN GROUPS 

In this section, we study the -Roo property for virtually abelian groups. These groups are 
finite extensions of abelian groups and they appear as fundamental groups of almost fiat 
Riemannian manifolds. 

The simplest torsion-free virtually abelian group with the -Roo property is the fundamental 
group of the Klein bottle K which is the total space of a S^-bundle over S^. On the 
other hand, K is finitely covered by the 2-torus so that tti{K) has as a finite index 
subgroup. Since does not have the -Roo property, it follows that in general the -Roo 
property is not even invariant under commensurability. Note that tti{K) is non-elementary 
non-Gromov hyperbolic and has polynomial growth. Furthermore, we will show that the 
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groups vri(i^) x Z" for every n > 0, which are infra-abehan having Z""*"^ as a subgroup of 
index 2, have the i?oo property. 

We will start by analyzing the group Z x Z where the action is non-trivial. This group is 
the fundamental group of the Klein bottle. The group of automorphisms of Z x Z is known 
as folklore. Since we cannot find a precise reference for this fact, we state it in the following 
lemma and we sketch a proof for completeness. 

Lemma 2.1. The group Aut(Z x Z) where ZxiZ is as above is isomorphic to (Z ^ Z2) x Z2 
where the action 6* : Z2 ^ Aut(Z0Z2) sends the generator to the automorphism (r, e) 
(— r, e) . Furthermore, the inner automorphisms are isomorphic to the subgroup (2Z ^ 0) x Z2 
and the quotient Out(G) is isomorphic to Z2 © Z2. 

Proof. We use the presentation («, /3|a/3a/3~^) for the fundamental group of the Klein bottle. 
Given an endomorphism k, write K{a) = f3' and k{(3) = a''(3\ Now, 

K,{aPaP^^) = K{a) hi{P) K{a) 
(2.1) =a'f3'a''(3^a'(3'{a^(3Y^ 

for some integer q that depends on the values of e, s,r, and 6. Since aPaP~^ = 1, the 
expression (12. ip yields the trivial element. Thus, 2s = and hence s = so that K,{a) = a*" . 
It follows that the automorphisms of this group are of the form a 1— > a^, /5 ^— fi^ . Moreover, 
e, 5g{1,— 1}. To see this, we note that given any two automorphisms a 1— > a^^, /5 1-^ a^^fi^^ 
and a ^ a^^,P ^ a^^P^^, the composite is an automorphism such that a 1— >■ a*^^"^^ and the 
/3-exponent of the image of /? is 6162- Consider the automorphisms 

ipi : a ^ a; /3 a/3 

Writing Z2 = {±1}, the generators for (Z © Z2) x Z2 are rji = (1, 1, 1), ri2 = (0, —1, 1) and 
?73 = (0, 1, —1). The homomorphism ipi 1— rji for i = 1, 2, 3 is the desired isomorphism. □ 

Now we will show that Z x Z has the Roo property. 

Theorem 2.2. For any automorphism (p ofZ xi we have R{(f) = 00. 
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Proof. Denote hy x,y the generators of the group Z xi Z, where x is a generator of the 
first copy of Z and y is a generator of the second copy of Z. In this group, which is the 
fundamental group of the Klein bottle, we have the relation xyxy'^ — 1. The automorphisms 
of Z XI Z from their description given in the proof of Lemma 2.1 can be divided into four 
cases as follows. 

a) x I— > x; y I— > x'^y b) x i— > x; y i— > x^y"^ 
c) X I— > x~^] y I— > x'^y d) X I— > x~^] y i— > x^y"^ 

where r G Z. 

Cases a) and c) are treated as follows. These automorphisms leave invariant the subgroup 
generated hy x. So every such automorphism induces a homomorphism of the short exact 
sequence 

o^z^z>^z^z^o 

and induces in the quotient the identity homomorphism of Z which has an infinite number 
of conjugacy classes. So the result follows. 

For Case b), we have the automorphism x — > x;y — > x^y~^ which maps a generic element 
rj-ruyk x^y"^ if k is even and to x^'^'^y'^ if k is odd (these elements are obtained by a 
straightforward calculation using the relation xyxy"^ = 1 on the group). So the elements of 
the group in the Reidemeister class of the element are of the form either x^y'^'^x'^y'^'^x"'^ 
or After simphfying the expression using the relation of the group, 

these elements can be written as x'y^" or respectively. So there are an infinite 

number of distinct Reidemeister classes since is in the Reidemeister class of x^ if and only 
— This proves Case b). 

For Case d), we have the automorphism x — > x~^] y — > x^y~^ which maps a generic element 
j.myk ^-ruy-k ^ jg evQu and to x~'^'^'^y~^ if k is odd. So the elements of the group in the 
Reidemeister class of the element x^y are the elements of the form either x^y'^'^x'^yy'^'^x^ or 
rj.my2n+irj.iyy2n+irj.m-r _ Again, using the relation of the group, these elements can be written 

as x'-y^^'^"' or a;''^*y^"+^, respectively. So there are an infinite number of distinct Reidemeister 
classes since a;*y and x^y are in the same Reidemeister class if and only iii = j. This proves 
Case d). □ 

Next, we construct from the Klein bottle examples of Bieberbach groups with the Roo prop- 
erty. First, the following result is crucial in our construction. 
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Proposition 2.3. The group Z xi Z2 has the Roo property. 

Proof. Let : Z xi Z2 ^ Z xi Z2 be an automorphism. Since tlie only elements of infinite 
order are the elements of the form f for t G Z a generator and r 7^ 0, the subgroup Z is 
characteristic. Therefore, ip\z : Z — > Z. Since Aut(Z) = {±1}, f\z is either the identity or 
multiphcation by —1. In the former case, the assertion that R{<f) = 00 follows from (2) of 
Lemma 11.11 

For the latter case, the Reidemeister class of (t', 1) contains at most two elements where 
1 is the non-trivial element of Z2. To see this, suppose [t^, 1) and {t^, 1) are (/^-conjugate, 
that is, (^^l) = a(r,l)^(a)"i for some a = {P,e) G Z x Z2 where e = 0, 1. Suppose 
!f{t^,l) = (r,l). Now, if (a) = (t-^+^",e) so that if{a)-^ is equal to (r-'+", 1) if e = 1 or 
if e = 0. It follows that 

a{f, l)ip{a)-' = {t\ l){f, l)(t~^+", 1) = (t^■-^ 0)(r^+", 1) = (^-^ 1) when e = 1 

and 

aif, l)ifi{a)-' = {t\ 0){f, 0) = (^^■+^ l)(t^ 0) = (f, 1) when e = 0. 

Since the set {(t',1)} is infinite and the Reidemeister classes have finite length, it follows 
that R{ip) = 00. □ 

The next result shows that for any positive integer n > 2, one can construct a finitely 
generated Bieberbach group of Hirsch length n that has the i?oo property. 

Theorem 2.4. The group tti{K) x Z" has the Roo property for all integer n > 0. 

Proof. Consider the presentation tti{K) = {a,b\abab^^) . The center of this group is the 
subgroup generated by and similarly the center of ttiIK) x is the subgroup (6^) x Z". 
Now consider the short exact sequence 

1 ^ (6^) X Z" ^ TTi{K) X Z" ^ Z X Z2 ^ 1. 

Since the center is characteristic with respect to automorphisms, any automorphism of 
7ri{K) X Z" is an automorphism of the short exact sequence. Since the quotient Z x Z2 
has the Roo property by Proposition 12. 3[ the result follows from Lemma 11.11 □ 



Remark 2.1. This result in the case n = gives an alternate proof of Theorem 12. 2[ 
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Remark 2.2. Given a virtually nilpotent group G, we have a short exact sequence 1 N 
G ^ F ^ 1 where F is finite and is nilpotent. If is characteristic and has the Roo 
property then it follows easily from Lemma [1.11 that G also has the Roo property. 

3. Commutators and free nilpotent groups 

Given r a positive integer and k either an integer or oo, consider the free nilpotent group 
G{r, k) = Fr/Tf:^i{Fr) of rank r and nilpotency class k, where for k = oo we have G{r, oo) = 
Fr because F^ is residually nilpotent. For which values of r and /c(including k = oo) does 
the group G have the Roo property? For the case k = oo, the free group Fr for r > 2 is a 
torsion-free non-elementary Gromov hyperbolic group thus it follows from [23] that Fr has 
the Roo property. We should point out that the proof in [23] is geometric. In this section, we 
will show that F2/Tg{F2) has the property. It seems that for i < 9 the group F2/Ti{F2) does 
not have this property. As a result, we will give a purely algebraic proof of the fact that an 
automorphism of a free group of rank 2 has an infinite number of twisted conjugacy classes. 

Let us denote r„(F2) simply by r„. From Witt's Formulae ({2S] Theorem 5.11), the ranks 
of the abelian groups Fj/Fj+i are calculated. In particular, we have: 

rA;(F2/F3) = 1, rkiT^/T,) = 2, rA;(F4/F5) = 3, rA;(F5/F6) = 6, rk{re/Tj) = 9, rA;(F7/F8) = 
18, rA;(F8/F9) = 30. 

Lemma 3.1. Let G{2,k) be the free nilpotent group of rank 2 on two generators a and b, 
nilpotency class k and ip : G{2, k) —>■ G{2, k) a homomorphism. Suppose k > 3. The induced 
homomorphism (p : F2(G(2, /c))/F3(G'(2, A;)) — > F2(G'(2, A;))/F3(G'(2, A;)) is multiplication by 
det(M) where M is the matrix of <p^^ : G{2,k)'^^ G{2,k)^^ and G{2,k)^^ denotes the 
abelianization of G{2, k) . 

Proof. Since A; > 3, it follows that Ti{G{2,k)) = Fj(F2)/Ffc+i(F2) for i < k. This means 
that F2(G(2, A;))/F3(G(2, k)) = F2/F3 and G{2, k)^^ = F2^^ Suppose the induced map 
is given by the matrix M = (° 5)- Note that F2/F3 is generated by the single element -BF3 
where B = [a,b] = aba~^b~^ . Using classical calculus of commutators (see [25] Chapter 5), 
we have -BaF3 = ai?F3 and BbV^ = 6-BF3. Since aba~^ = Bb, it is straighforward to verify 
that p{BT^) = B'Ts where k = ad - = det(M). □ 

Corollary 3.2. If if : G{2,k) G{2,k) is an automorphism such that det(v9'^'') = 1 then 
R{(p) = 00. 
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Proof. We have a homomorphism of short exact sequence r2/r3 G{2, k)/r^ 
G{2,k)/T2 0. Since the induced map on the subgroup r2/r3 = Z is multiphcation by 
1, it follows from formula (2.2) of [15] that G'(2, A;)/r3 has the Roc property. Since is 
characteristic in G{2, k), the result follows for G{2, k). □ 



In order to study the i?oo property, it suffices, as a result of Corollary 13. 2[ to consider 
automorphisms (p whose induced automorphisms ip^^ have determinant —1. While we have 
succeeded in determining the i?oo property for G{2, k) for > 9, we do not know how to 
extend our techniques to G{r, k) for r > 2 and k > 2. On the other hand, the case of the 
free abelian group G{r, 1) of rank r is well-understood. 

Next, we will construct a nontrivial element in Fg/rg C G{2, 8) which is fixed by the restric- 
tion of any automorphism ip : G{2, 8) G{2, 8). 

Proposition 3.3. //(/? : G(2,8) G{2,8) is an automorphism then R{ip) = oo. 

Proof. By Corollary 13. 2^ the result holds if det{ip^^) = 1. So let us assume that det{{p^'^) = 
— 1. Then consider the quotient r3/r4. This group has rank 2 and has as generators 
[-B, x], where B = [x,y]. A straightforward calculation shows that the matrix of 

the automorphism induced by (f has determinant -1. Therefore the image of the element 
[[B,x], [B,y]] G Tq/Tj is ([[5, a;], [B,y]])~'^. Call this element w. From Lemma 5.4 (pg 
314), it follows that w 1. Now consider the element Wi = [B,w] G Tg/Fg. From above, 
we have v^'(wi) = wi where is the induced homomorphism on Fg/Fg and again from |25j 
Lemma 5.4 we conclude that Wi ^ 1. Then the automorphism ip induces an automorphism 
(p of the following short exact sequences and hence a commutative diagram. 

> Fg/Fg > Fa/Fg > Fa/Fg > 1 



> Fg/Fg > Fs/Fg > Fa/Fg > 1 

If the induced map on the quotient has infinite Reidemeister number then the result follows. 
Otherwise, it will be finite. Note that y^'iwi) = wi and Fg/Fg is finitely generated torsion- 



free abelian. Applying Lemma [1.11 we have R{(p') is infinite and together with the fact that 
R{<f) is finite, we conclude that R{ip) is infinite and the result follows. □ 

Since the commutator subgroups F^ are characteristic, we can use induction and the same 
argument as in the proof above to the short exact sequence — >• F^/F^+i F2/Ffc+i — > 
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F2/Tk ^ 1 to show that F2/Tk+i has the desired property. Thus, we state the following 
result. 

Theorem 3.4. Let G{2, k) he the free nilpotent group on 2 generators of nilpotency class k. 
If k > 9 (including the case k = oo) then for any automorphism ip : G{2,k) G{2,k), we 
have R{ip) = oo, i.e., G{2, k) has the R^o property. 

Remark 3.1. When k = oo, G(2, oo) = F2. Thus, our proof of Theorem 13.41 provides a purely 
algebraic proof of the fact that F2 has the _Roo property as a consequence of a result proven 
in using techniques from geometric group theory. 

4. Roo AND NILPOTENCY CLASS 

If G is a finitely generated torsion-free abelian group, then G does not have the R^o property. 
Since abelian groups have nilpotency class 1 and Example 11.21 gives an example where the 
nilpotency class is 6, it is interesting to know if one can construct a nilpotent group of 
nilpotency class < 5 that has the desired R^o property. First, is there a finitely generated 
torsion-free nilpotent group of nilpotency class 2 which has the Roo property? 

Example 4.1. We now construct a group of nilpotency class 2 which has the Roo property. 
Consider the free nilpotent group G(4, 2) on 3 generators x, y, z, w of nilpotency class 2 
and r2(G'(4, 2))/r3(G'(4, 2)) is a free abelian group of rank 6 in which the cosets defined by 
the commutators [x, y], [x, z], [x, w],[y, z], [y, w], [z, w] form a basis. Take the quotient by the 
subgroup generated by [x, z], [x, w], [y, z]. We will prove that this group has the Roo property. 

Let if : -Fli/Fs — ^ -^4/r3 be an automorphism such that the matrix on the abelianization is 
given by 

ai 02 ^3 04 
^ _ 61 62 h h 

Cl 02 C3 C4 
di (^2 '-^3 '^^4 

The proof is divided into several steps. 

Step 1. We will show that bi = di = = d^ = 0. Suppose that 63 7^ 0. The proof of the 
other cases is similar and we leave to the reader. Because the determinant of each of the 
2x2 matrices, 
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bi bj 




Ci Cj 


bi bj 




di dj 




di dj 



for — (1, 3), (2, 3), (1, 4) is zero we have (01,03) = A(6i,&3), (02,03) — Xi{b2,bs) so that 
A = Ai- Similarly {di, d2, d^) — Q;(fei, 62, ^'3)- Hot — then d^^ since the determinant of the 
matrix is nonzero. But 01^4 = 61^4 = which implies that ci — bi — 0. So the first column 
of M is zero, a contradiction. So let a 7^ 0. Then similarly because abs ^ 0, it follows that 
(ci, C2, C3) = ^(61, 62, ^s)- So the rows of M up to the third column are proportional and the 
determinant is zero. This is a contradiction and thus 63 = 0. 



Step 2. From now on, we will consider that 61 = rfi = 63 = ^3 = 0. In this step we will 
compute the form of the matrix under the assumption that 03 7^ 0. Because 61 = di = 63 = 
(i3 = we must have 62O3 = d^cz = 0164 = Cid4 = 0. Since 03 7^ implies 62 = 0, 64 7^ 0, 
and Oi = 0, the matrix M is of the form 

02 03 04 
64 

M = 

Ci C2 C3 C4 
^2 ^4 



Now det(M) = 640301^2- Because this determinant is ±1, it follows that all the factors 
are either 1 or -1. Also following from the equations above, we have C3 = ^4 0. Thus 
det(M - Id) ^1- b4d2 - C1O3 + fo4(^2Ci03. If det(M - Id) ^ then we have R{(p'^'') = 00. 
This always happens except when 64(^2 = C1O3 = — 1 in which case det(M — Id) — 4. In this 
situation, we are going to compute the matrix N of the induced homomorphism on r2/r3. 
This matrix is 






Cid2 



O264 0364 
-(^264 

—d2C4 



Using the fact that (i2&4 
and the result follows. 



— 1, we have det{N — Id) = since the second row becomes trivial 
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Step 3. Suppose now that 03 = 0. Then we have 



M 



ai a2 04 
62 64 



Cl C2 C3 C4 



d2 



So we have d2Cs = aib^ = cid^ = 0. Since C3 7^ 0, it folows that d2 = 0. Since det(M) = 
cti^2C3(i4, it follows that all factors are ±1 and consequently 64 = Ci = 0. Then det(M — Id) = 
(ai — 1)(62 — l)(c3 — l)((i4 — 1) and as before if det (M—Irf) = then the Reidemeister number is 
infinite. This always happens except when ai = 62 = C3 = ^4 = — 1 in which case det (M—Irf) 
is 16. In this case we are going to compute the matrix N of the induced homomorphism on 
r2/r3. This matrix is 



Using the fact that ai = b2 = —1 we have det(A^ — Id) = since the first column becomes 
trivial and the result follows. This concludes the proof. 

In contrast to the last example, we will next show that the free nilpotent groups of nilpotency 
class 2 do not have the -Roo property. 

Example 4.2. We now show that the free nilpotent groups G{r, 2) does not have the i?oo 
property. It suffices for each r to exhibit an automorphism (f : G{r, 2) G{r, 2) such that 
R{f) < 00. 

For r = 2, consider the automorphism given by (p{x) = x^y and (p{y) = x^y"^. The matrix of 
the induced automorphism on the abelianization Z©Z has two eigenvalues Ai = 2+5^^^, A2 = 
2 — 5^/^. It follows that Fix{ip^^) = {1} so that R{lp^^) < 00. The induced automorphism ip' 
on r2/r3 = Z is multiplication by —1 so that R{y:>') < 00. Since we have a central extension 
we have R{(p) = R{(p^'')R{ip') by Lemma [TTT] and the result follows. 
Let r > 3. If r is even, define ^ : (Z©Z)'^/2 ^ (Z©Z)^/2 

as a direct sum of the automorphism 

y9:Z©Z^Z©Z defined above. If r is odd, define 



N 



aih2 -62^4 
62 C4 

02^4 



03^4 











Lp:{Z® Z)('^-i)/2 © Z ^ (Z © Z)("-i)/2 ^ 2 
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as a direct sum of the automorphism <^:Z©Z— s>Z©Z defined above and the automorphism 
given by multiphcation by -1 in the last coordinate. It is easy to see that the product of any 
two eigenvalues of (p for either r even or r odd cannot be 1. Therefore if' : r2/r3 r2/r3 
does not have 1 as an eigenvalue so that Fixip' = {1} and the result follows from Lemma 

o 

5. Roo AND HiRSCH LENGTH 

There is only one finitely generated torsion-free nilpotent group of Hirsch length 2, namely 
Z©Z and it does not have the i?oo property. For which integers n, is there a finitely generated 
torsion-free nilpotent group of Hirsch length n with the -Roo property? 

Example 5.1. Let us consider nilpotent torsion-free groups of Hirsch length 3. They are 
classified as follows. For each integer r, consider the group A^,- given by the following pre- 
sentation (a, 6, c|[a,c] = 1, [6, c] = 1 and [a,b] = d). It is not hard to show that there is 
an automorphism (f : Nr ^ Nr defined by the map which sends a o?h and h — > a^lP' . 
The automorphism (p induces a commutative diagram of automorphisms of the short exact 
sequence 

o^c^A^^^zez^o 

where C is the center of the group. Since the extension is central and the induced automor- 
phisms if' and Tp have finite Reidemeister numbers, it follows from (4) of Lemma 11.11 that 
R{'p) is finite. 

Example 5.2. We now construct a finitely generated nilpotent group G of Hirsch length 4 
such that r4(G) = 1, r3(G) = Z, and G has the -Roo property. This group is a quotient of the 
nilpotent group -F2/r4(-F2) where F2 is the free group on the generators. The group G has 
the property that G/V'^iG) = F2/V^{F2). The difference comes between r3(G)/r4(G') = Z 
and r3(F2)/r4(F2) = Z © Z (see e.g. |25j). Now we describe G in terms of generators and 
relations. Let x,y be the generators, B = [x,y] and wi = [[x,y],x] such that = 1. 

Note that the last relation does not hold in the group -F2/r4(-F2) . Now given an automorphism 
of G, the matrix of the abelianization has a special form since [B,y] = 1. This matrix has the 
first column a, b and second column 0, d. As a consequence, the automorphism induced on 
the abelianization either has eigenvalue 1 or det{ip) = 1. In the former case the result follows 
because the induced map on the abelianization has Reidemeister number infinite. In the 
latter case the result follows from an argument similar to Lemma 2.1. Let H = F2/T i{F2). 
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Then the group G given by the presentation (x, ?/|r4, [B,y\), which can be regarded as a 
quotient of if, has the R^o property. 

Remark 5.1. Example 15 . 21 does not imply that F2 has the i?oo property but it gives an example 
of a nilpotent group with the -Roo property that is simpler than F2/Tq{F2) . Further, it has 
the least possible Hirsch length among all finitely generated torsion-free nilpotent groups 
having the -Roo property. 

Example 5.3. Here we construct for every integer £ > 4 a finitely generated torsion- free 
nilpotent group of Hirsch length £ with the -Roo property. We will show that the product of 
the group given in Example 15.21 with IP' for n > 0, has the desired property. 

Now for any n > 0, let Gn = G x Z" where G = {x,y\r4, [B,y]) is the group constructed in 
Example 15. 2[ Then, TkiGn) = ^kiG) so Gn is of rank n + 2, has nilpotency class 3, and its 
Hirsch length is n + 4. Moreover, r3(G„) C Z{Gn) = Z{G) x Z". Now, given G Aut(G„), 
we have the following commutative diagram. 



(5.1) 



T^{Gr. 

r3(G„) 



Gr. 



Gr. 



GIV^,{G) = GjT.iG,, 
G/T,iG) = GjT.iGn) 



Since G has the -Roo property and the following commutative diagram 



(5.2) 



G 



G 



G/T,{G) 
c 

G/TsiG) 



yields the product formula -R(C) = R{C)R{() by (4) of Lemma [TTT| it follows that 

R(Q = 00 ^ i?(C') = 00 or R{C) = 00. 

Thus R{(p) = 00 if -R(C) = 00. If -R(C') = 00 then R{(p') = 00 since T^^Gn) projects onto 
r3(G). Hence, R{(p) = 00. 
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6. Fixed point free homeomorphisms on nilmanifolds 



Recall that a compact nilmanifold is the coset space of a finite dimensional nilpotent Lie 
group by a closed cocompact subgroup. A classical result of A. Mal'cev asserts that such 
nilmanifolds are coset spaces of simply connected nilpotent Lie groups by uniform discrete 
subgroups. Furthermore, there is a one-to-one correspondence between finitely generated 
torsion-free nilpotent groups F and compact nilmanifolds M = K{r, 1). 

From Example 15. 3[ we can associate to each such finitely generated torsion-free nilpotent 
group a compact nilmanifold so that the Reidemeister number of each homeomorphism is in- 
finite. As an application, we obtain the following result for fixed point free homeomorphisms 
on these nilmanifolds. 

Theorem 6.1. For any n > A, there exists a compact nilmanifold M with dimM = n such 
that every homeomorphism f : M ^ M is homotopic to a fixed point free map. Moreover, if 
n > 5 then f is isotopic to a fixed point free homeomorphism. 

Proof. For any n > 4, let M be the compact i^'(G„-4, 1) nilmanifold where Gk denotes the 
group as constructed in Example 15.31 Since the Hirsch length of G„_4 is n, it follows that 
dimM = n. If f : M —>■ M is a. homeomorphism, then is an automorphism of Gn-i- 
Thus, R{f#) = oo. It follows from \1Q\ or [I3l that A^(/) = 0, that is, the Nielsen number 
vanishes. It follows from a classical theorem of Wecken that / is deformable to be fixed point 
free. If n > 5, we invoke the isotopy Wecken theorem of Kelly [21j to obtain the desired 
fixed point free homeomorphism in the isotopy class of /. □ 

Remark 6.1. The isotopy Wecken theorem of Kelly also holds in dimension 2 [20] based 
upon Nielsen-Thurston classification of surface homeomorphisms and in dimension 3 [19j 
based upon techniques of 3-manidols. It is however unknown whether it holds in dimension 
4 in general. In fact, it is not even known whether a homeomorphism of the 4-torus with 
zero Lefschetz number (hence deformable to be fixed point free) can be isotopic to a fixed 
point free homeomorphism. 

It is known [26j that the well-known Arnold conjecture holds for nilmanifolds which states 
that the minimal number of fixed points of Hamiltonian symplectomorphisms of a compact 
nilmanifold of dimension 2n is at least 2n + 1 (for more details on the Arnold conjecture, 
see e.g. [2^). Based upon our results, any symplectomorphism of the even dimensional 
nilmanifolds constructed in Theorem 16 . 1 1 can be isotoped to a fixed point free diffeomorphism 
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but the isotopy does not respect the Hamiltonian structure so that the resulting fixed point 
free diffeomorphism is not a Hamiltonian symplectomorphism. 

Remark 6.2. It is well-known (see e.g. [3] or PU]) that there is a one-to-one correspondence 
between compact flat manifolds and flnitely generated Bieberbach groups, that is, compact 
Riemannian manifolds with zero sectional curvatures are precisely the flnite dimensional as- 
pherical manifolds with flnitely generated torsion-free virtually abelian fundamental groups. 
It is natural to ask whether one can obtain a result similar to Theorem 16. II for flat manifolds. 
One of the main differences between nilmanifolds and infra-nilmanifolds (or flat manifolds) 
is that A^(/) = iff R{f) = oo for selfmaps on nilmanifolds whereas there are maps on flat 
manifolds where R{f) = oo but N{f) > (see Example 5.3 of [16]). Even for the Klein 
bottle K, one can flnd a homeomorphism with A^(/) > and R{f) = oo. Thus, the groups 
^ilK) X Z" do not provide examples of flat manifolds on which all homeomorphisms have 
zero Nielsen numbers. 

A natural extension of our results for nilmanifolds is that of an infra-nilmanifold whose 
fundamental group is a flnitely generated torsion-free virtually nilpotent group or equiva- 
lently an almost Bieberbach group. From the point of view of Nielsen flxed point theory, 
D. Anosov in [Ij (or [5J) already pointed out that the equality N{f) = |i^(/)| does not 
hold for selfmaps of infra-nilmanifolds. Furthermore, Kwasik and Lee [22] constructed infra- 
nilmanifolds and affine Anosov diffeomorphisms / for which |i^(/)| 7^ N{f) in every even 
dimension n > 4. On the other hand, for a selfmap / of compact solvmanifold, R{f) < 00 im- 
plies that A^(/) = R{f) tlSj- Thus, one would like to study the Roo property for such groups, 
although we cannot always obtain results similar to those in the previous two sections. 

7. C-NILPOTENT GROUPS 

Next we turn to another generalization of nilpotent groups. Nilpotent spaces have proven to 
be useful in homotopy theory and in algebraic topology in general. The concept of a Serre 
class of abelian groups has been generalized. One such generalization is that of a C-nilpotent 
class, flrst introduced in [14j. A family C of groups is a class of groups if given a short exact 
sequence of groups 

1 B ^ C ^ 1 

then A, C G C if and only if B E C. Given a class C, a group G is said to be C-nilpotent if 
Tn{G) G C for some positive integer n where Ti{G) is the i-th commutator in the lower central 
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series of G. More generally, a group vr acts on a group G C-nilpotently if T'^{G) G C for some 
positive integer n where rj^(G) is the smallest vr-subgroup that contains [G^T^_^{G)] and 
the set {(a ■ g)g~^\a E n,g E r^_]^(G')}. Thus, a space X is said to be C-nilpotent if vri(X) 
is C-nilpotent and the action of vri(X) on 7Tk{X) is C-nilpotent. From now on, we let C be 
the class of finite groups. 

Let G be a finitely generated group. For any (f G Aut(G) and any positive integer n, we 
have a commutative diagram 

> r„(G) > G > G/r„(G) > 1 

(7.1) 

. r„(G) > G . G/r„(G) . 1 

It follows easily from Lemma 1 1.11 that if G/r„ (G) has the i?oo property then so does G. 
Since A^„ = G/Tn{G) is nilpotent, the torsion elements form a subgroup T„ and T„ is a 
characteristic subgroup of G/Tn{G). It follows that if Nn/Tn has the i?oo property then 
also has the R^o property. 

Thus, we have the following useful result that can be used to construct more examples of 
groups with the -Roo property. 

Proposition 7.1. Let G be a finitely generated group and he the torsion subgroup of the 
nilpotent group Nk = G/Vk{G). If for some positive integer n the group Nn/Tn has the Roo 
property then G also has the Roo property. 

We now combine our results from section 4 to enlarge the family of groups with the -Roo 
property and also the family of compact manifolds which have fundamental groups with the 
-Roo property. 

Example 7.2. Let S'^ be the Poincare 3-sphere with 7ri(S'^) = Icos, the binary icosahedral 
group of order 120. Suppose M is the 4-dimensional -ft'(vr, 1) nilmanifold where vr is the 
torsion-free nilpotent group of nilpotency class 3 as in Example 15. 2[ Take X = T,^ x M. 
Now, G = 7ri(X) is C-nilpotent, r4(G) = Icos, and G/T^IG) = vr. Since tt has the -Roo 
property (and the torsion subgroup of vr is trivial), it follows from Proposition 17.11 that G 
has the -Roo property. More generally, one can obtain similar examples by replacing Icos 
with any finite perfect group. 
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We can show more for homeomorphisms of the space X in the last example although it is not 
clear if X is a Jiang- type space. In fact, we obtain the following result, similar to Theorem 



Theorem 7.3. For any n > 7, there exists a compact C-nilpotent manifold M with dim M = 
n such that every homeomorphism f : M ^ M is isotopic to a fixed point free homeomor- 
phism. 



Proof. Let n > 7 and M = S'^ x A^"^'^ where A^^ is the compact nilmanifold with fundamental 
group of Hirsch length k as constructed in Example I5.2[ We will show that for any / : 
M — > M, / is homotopic to a fiber-preserving map. First, choose basepoints Xq G S'^, 
yo E N = N"-"^ and write f{x,y) = {g{x,y), h{x,y)). Denote hj f : N ^ N the restriction 
of h on {xq} X A^, that is, f{y) = h{xo,y). Next, we will show that f o P2 and h are 
homotopic where p2 : ^ N ^ N is the projection on the second factor. Write A = 
X N X {0, 1} U (S^ V M) X [0, 1]. The pair {M,A) is a relative CW complex. Since 
7ri(S^) = Icos is finite, A^ is aspherical and tt = 7ri(A^) is torsion-free, the maps h and f op2 
restricted to are nuU-homotopic and they are homotopic on {xq} x A^. It follows that 
h and f o P2 coincide, up to homotopy, on the subspace V A^ C S'^ x A^. Let F be the 
homotopy from /i|E3vAf to {fop2)\^^yN and F : A N where F\mx{q} = h,F\Mx{i} = f°P2 
and -FUx[o.i] = F. Since the 2-skeleton of M = S'^ x A^ already lies inside T,^ V N G A, there 
is a sequence of obstructions d{F) G H^{M, A; 7rj_i(A^)) to extending F to M where i > 2. 
These obstructions are trivial because TTk{N) = for k > 1 since A^ is aspherical. Now, up to 
homotopy, the map h is of the form f o P2 thus we have the following commutative diagram 



M M 



(7.2) 



P2 



P2 



N — ^ N 

Now, suppose / is a homeomorphism and fl7.2l) holds. If / does not have any fixed points 
there is nothing to prove. Suppose / does have a fixed point then (17. 2p yields a commutative 
diagram of groups similar to the diagram as in Lemma [TTTl Thus, / induces an automorphism 
(f which has R{<f) = oo since vr has the Roo property. It follows that A^(/) = 0. Since the 
fibration p2 is trivial, it follows that A^(/) = N{f') ■ N{f) = where /' is the restriction of 
/ on the fiber. Since n > 7, Kelly's Isotopy Wecken Theorem [21] implies that / is isotopic 
to a fixed point free homeomorphism. □ 
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Remark 7.1. Although the space M in Theorem 17.31 is a C-nilpotent space, its fundamental 
group has a center of infinite index so we cannot conclude from [T7| that M is of Jiang- type. 
We should point out that one can construct similar manifolds M with the desired property 
by replacing with any compact manifold L with finite HiiL) since the same argument 
also shows that every map f : L x N ^ L x N is fiber-preserving up to homotopy. Note 
that if dimL < 2, vri(L) is necessarily abelian so that vri(M) would be nilpotent possibly 
with torsion. 

Remark 7.2. It is shown in [T7j that if if is a finite subgroup of a compact connected Lie group 
G, then the coset space X = G/H is C-nilpotent and its fundamental group has a center of 
finite index and hence is of Jiang-type. However, these spaces do not provide examples like 
Example 17.21 because [7ri(X), 7ri(X)] is also finite. It follows that 7ri(X)/[7ri(X), 7ri(X)] = 
7ri(X)^^ is finitely generated abelian and therefore does not have the Roo property and hence 
Proposition 17.11 is not applicable. 
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